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Abstract 

The  investigation  extended  previous  research  on  performance  tost3  in  ele- 
mentary collogo  physics.  The  effect  of  high  school  physics  on  colloge  physics 
achievement  was  explored.  Correlation  coefficients  between  performance  tests  and 
other  measures  of  aptitude  and  ach  lavement  wore  calculated . The  dependence  of  the 
test  reliability  on  tho  number  of  tost  items  and  subscores  was  investigated. 

It  was  found  that: 

1.  High  school  physics  background  probably  contributed  favorably  to 
achievement  in  college  physics  as  measured  by  conventional  examinations,  but  tho 

of feet  on  performance  tests  scores  wa3  doubtful. 

2.  The  correlations  betweon  the  performance  to3t,  ACE  and  high  school 
rank  wore  not  significantly  different  from  zero. 

3.  Tho  correlations  between  the  performance  test  and  conventional 
achievement  criteria  in  college  physics  wore  positive,  significant,  but  very  low. 

4.  The  Hoyt  reliability  coefficient  of  a performance  test  could  be 
increased  to  an  acceptably  high  value  by  entoring  several  subscores  for  each  item. 

Performance  Teste  in  Physics* 

This  study  is  an  extonsion  of  the  previously  reported  research  on  perform- 
ance tests  in  physics  (3,  4).  During  tho  spring  quarter  1951-52  additional  samples 
voro  drawn  frean  the  general  courses  In  physics  at  the  University  of  Minnesota.  Tho 
offoct  of  high  school  physics  on  achievement  in  college  physics  was  furthor  investi- 


* Under  contract  N8onr-66213  with  the  Office  of  Naval  Research,  Project  NR153-1^8 


go. tod;  a number  of  changes  were  made  in  tho  performance  tests;  some  of  the  original 
findings  wore  chocked;  and  tho  test  reliability  was  explored  more  thoroughly. 

Effect  of  High  School  Physics 

Tho  tost  results  for  Physics  f,  fall  quarter  1951?  showed  no  significant 
differences  on  three  achiovcnont  measures  between  students  who  had  had  high  school 
physics  and  those  who  had  nono  (3)-  The  anlysis  was  repeated  for  a comparable  sample 
drawn  frem  two  courses  offered  during  spring  quarter  1952.  It  is  apparent  from  Table 
1 that  the  conclusions  from  the  socond  investigation  appear  to  contradict  the 
original  findings. 

There  was  a significant  difference  in  means  for  tho  spring  quarter  group 
for  the  final  course  grade,  Cooperative  Fhysics  Test  for  Collogo  Students  and  tho 
laboratory  performance  test  in  favor  of  those  who  had  had  high  school  physics.  This 
difference  between  tho  two  groups  could  hardly  be  attributed  to  the  fact  that  tho 
fall  quarter  course  contained  largoly  sophomores,  whereas  tho  spring  quarter  sample 
was  composed  primarily  of  third  quartor  freshmen.  Both  groups  - freshmen  and  sopho- 
moros  - were  adjudged  to  be  random  samples  from  a normal  population  on  tho  basis  of 
throe  initial  measures ; High  School  Percentile  Rank,  ACE,  and  Cooperative  Tost  in 
High  School  Fliysic6. 

When  the  subsamplos  from  both  quarters  wero  poolod  the  moans  wore  still 
significantly  different  for  the  final  grade  and  the  cooperative  test.  Sinco  four  of 
the  nine  items  on  the  performance  test  wore  identical  and  since  the  means  for  the  two 
quarters  differed  by  only  one  point,  it  was  thought  reasonable  to  pool  all  the  stu- 
dents on  this  test  and  compare  those  who  had  had  high  school  physics  with  those  with- 
out it.  It  was  found  that  there  wero  no  significant  differences  in  the  variances  or 
the  means.  However,  the  number  of  students  ontering  engineering  physics  at  the  Uni- 
versity of  Minnesota  without  high  school  physics  is  relatively  small:  10  to  lp  per 


Achievement  of  Students  with  and  without  High  School  Fhysico 
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cont  in  our  samples.  Consequent ly,  the  data  on  the  contribution  of  high  school 
physics  must  be  interpreted  with  great  caution.  Hie  h igh  correlation  coefficients 
botween  a pre-test  in  high  school  physico  and  the  theoretical  achievement  measures 
support  the  hypothosis  that  a high  school  physics  course  has  probably  a favorable 
offect  on  achievement  in  college  physics  as  measured  by  the  usual  criteria.  (Gee 
Table  2)  On  the  other  hand,  from  thG  low  correlation  coefficient  betwoen  tho  came 
pT'o-teot  and  the  performance  test  a reasonable  conclusion  is  that  tho  influence  of 
r.igh  school  physics  on  laboratory  performance  tc3t  scores  is  not  likely  to  bo  Big- 
nix' leant. 

additional  evaluation  will  bo  needed  to  demonstrate  conclusively  the  effect 
of  high  school  physics  on  the  theoretical  and  performance  achievement  in  college 
phye ics. 


Modification  of  forformance  Tests 

Experience  with  performance  tests  during  1951-52  led  to  several  changos  in 
tho  design,  administration,  and  scoring  of  test  items.  More  items  with  a high  por- 
foimanco  component  woro  introduced.  Trie  credit  allocation  for  the  theoretical  phase 
of  each  itom  was  reduced  or  eliminated;  thus  no  advantage  accrued  to  groups  taking 
tho  test  at  a later  time  because  of  information  leakage  on  method . Each  itom  was 
graded  by  ono  instructor  for  all  the  students  in  the  course.  One  week  before  the 
tost  oach  student  was  givon  a shoot  on  which  the  tost  itoms  woro  described  in  torms 
of  the  given  apparatus  and  tho  problom  to  be  solved  or  measurement  to  be  made.  Thus 
every  student  was  given  tho  same  opportunity  to  make  a theoretical  preparation  for 
tho  tost.  During  the  tost  all  the  needed  formulas  wore  either  written  on  the  board 
or  were  available  on  a mimeographed  shoot. 

The  offect  of  the  modified  testing  proceduro  was  investigated  during  spring 
quarter  1952.  A samplo  of  61  Institute  of  Technology  students  was  drawn  from  Ihysics 
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7,  taught  by  Frof.  J.  Valasek.  This  group  was  given  substantially  the  sane  Instruc- 
tional expoouro  as  the  subjects  on  which  the  original  findings  were  based  (3)*  Ihe 
means,  standard  deviations,  and  correlations  coefficients  for  the  two  groups  are 
shown  in  Table  2.  It  is  clear  that  the  means  and  the  standard  deviations  on  the 
initial  and  final  measures  do  not  differ  greatly  bo tween  the  two  samples.  The  only 
exception  is  the  standard  deviation  on  the  performance  test  which  incroasod  by  a 
factor  of  two  for  the  spring  quarter  group. 

Hie  correlation  coefficients  between  the  initial  measures,  between  the 
initial  and  final  measures  and  the  theoretical  criteria  did  net  differ  substantially 
from  the  corresponding  values  for  tho  fall  quarter.  However,  the  correlation  co- 
efficients between  the  spring  quarter  laboratory  performance  test  and  all  the  other 
measures  have  been  decreased  to  about  half  of  the  corresponding  coefficients  for  the 
fall  quarter  group.  For  instance,  the  correlation  between  the  performance  score  and 
High  School  Percentile  Sack  has  been  reduced  from  .33,  significant  at  one  per  cont, 
to  .?C,  which  is  not  significantly  different  frem  zero.  Hio  sane  is  true  of  tho 
correlation  coefficient  between  performance  scoros  and  ACE.  Apparently  the  aforemen- 
tioned changes  in  the  performance  tost  account  for  tho  reduction  in  the  size  of  the 
coefficients.  The  highest  correlation  is  .33;  between  the  performance  tost  and  the 
final  grade  in  the  course.  Although  the  coefficient  is  significantly  different  from 
zero  at  tho  one  per  cent,  its  low  value  is  indicative  of  the  difference  in  the  abil- 
ities and  skills  sampled  by  each  of  the  two  measures.  In  addition  there  is  some 
intercorrelation  betweon  the  two  measures  since  the  performance  test  accounts  for 
about  7 per  cent  of  the  final  grade.  The  low  positive  correlation  is  simply  a 
measure  of  the  overlap  in  terminology,  and  apparatus  between  the  laboratory  work  and 


lectures . 


Item  Analysis 


Tne  Davis  technique  (l)  was  used  to  find  the  discrimination  and  difficulty 
indices  of  the  performance  test  in  Physics  7 for  spring  quarter  1952.  The  results 
are  shewn  in  Table  3*  Four  of  the  nine  items  were  identical  with  those  given  during 
the  fall  quarter.  The  criteria  used  vero  the  total  score  on  toe  tost  and  the  final 
course  grade.  The  average  discrimination  ar.d  difficulty  index  for  the  entire  test 
was  practically  unchanged  for  too  total  score,  "he  discrimination  irdex  was  some- 
what lower  for  toe  spring  group  with  the  final  grade  as  a criterion.  However,  for 
the  items  common  to  both  quarters  the  average  discrimination  and  difficulty  indices 
were  higher  on  both  criteria  for  the  spring  group.  Hence  the  modifications  in  the 
performance  test  resulted  in  items  that  on  the  average  discriminate  better  between 
the  good  and  poor  students  and  coco  closer  to  too  optimum  difficulty  of  50  per  cent. 

Hoyt  Reliability 

The  Hoyt  reliabilities  (2)  of  the  performance  test  were  computed  for  the 
long  items,  short  items  and  too  total  test.  The  values  for  the  fall  and  spring 
quarters  are  shown  in  Table  4.  It  is  seen  that  toe  reliabilities  are  highly  compar- 
able. The  largo  difference  in  the  reliabilities  cf  the  long  and  short  item  parts 
suggested  an  investigation  of  the  test  reliability  a3  a function  of  the  number  of 
parts  in  a test. 

During  the  fall  quarter  of  1952  the  graders  of  the  performance  tests  were 
aaked  to  record  toe  credit  allocations  for  each  part  of  a given  item.  For  example, 
in  one  item  the  subject  was  asked  to  find  the  acceleration  ard  the  average  velocity 
of  a tall  rolling  down  an  inclined  plane.  'Ihe  credit  allocation  for  a perfect  score 
^as:  1 point  for  the  measurement  cf  the  plane's  length,  2 points  for  the  time 

doterminat ton,  2 points  for  the  correct  acceleration  valus,  and  2 points  for  the 
correct  velocity  value. 
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Table  3 

Item  Analysis  of  the  Laboratory  Performance  Tost 
University  of  Minnesota  - rhysico  7 


Sub3ample  12  - Spring  1952  - S3  12 
Subsample  9 - Fall  1951  (in  parentheses) 


Short  itom  test 
Item  Wo. 

Indl 

Discrimination 

Criteria** 

1 2 

ces* 

Difficulty 
Criteria** 
1 2 

SS . 12 
N = 8l 

Item  A vc. 

* 

All  Ihys.  7 
N = 221 

M-138  a 

38(55) 

25(11) 

46(31) 

42(36) 

35 

33 

M-106  m 

56(38) 

37(33) 

40(38) 

44(35) 

31 

29 

M-71 

27(14) 

23(1) 

25(31) 

26(30) 

12 

12 

M-36,  31b 

43 

10 

28 

29 

15 

13 

M-222 

13 

17 

53 

53 

53 

51 

H-2 

12 

14 

59 

59 

63 

63 

E-93 

11 

10 

55 

54 

63 

65 

S.  Avorago 

28 

20 

44 

44 

39 

38 

Long  Item  Test 

M-151 

37(41) 

10(12) 

52(51) 

47(51) 

46 

46 

M-79  m 

43 

12 

50 

54 

58 

60 

L.  Aver ago 

40 

11 

51 

51 

52 

55 

S.  & L.  Average 

31 

18 

45 

45 

45 

45 

Avorage  of  Common 
Items  SS.  12  Sc  9 

40(32) 

25(14) 

41(38) 

40(33) 

* Davis  Indices 


**  1 = Total  score  on  laboratory 
performance  te3t 
2 = Final  grade  in  physics  7 


o ^ 

A sample  of  62  Physics  7 students  vas  drawn.  The  subjects  had  the  cane 
characteristics  as  those  of  the  previous  two  quartors.  Tho  Hoyt  reliability  of  the 
performance  tosts  for  this  group  was  found  in  two  ways:  by  using  tho  total  score  on 

each  item,  and  by  repeating  tho  calculations  with  part  scoros.  In  the  first  ca3o 
there  were  10  numbers  for  oach  individual  grade;  in  the  socond  trial  there  woro  30 
parts.  The  Hoyt  reliability  for  the  first  procedure  was  .60,  which  was  vory  nearly 
the  3ane  as  that  for  the  spring  quarter  group.  On  tho  other  hand  tho  coefficient 
went  up  to  .75  whon  the  subdivided  parts  woro  used.  Tho  process  was  ropoated  with 
two  samples  of  studonts  from  a different  course. 

Two  random  samples  of  62  and  64  malo  studonts  woro  ri ravr.  from  Physics  la, 
fall  quarter  1952.  The  Hoyt  reliability  was  computed  with  6 items  of  a perfcraanco 
test  as  well  as  with  the  21  parts  of  tho  items.  Tho  coefficients  were  .44  and  .43 
for  tho  first  method  and  .89  and  .93  for  the  second  scheme  as  shown  in  Table  4. 

There  appoaie  to  be  sane  very  strong  evidence  in  support  of  the  relationship  between 
the  Hoyt  reliability  and  tho  number  of  part3  or  items  in  a performance  test.  Tho 
distribution  of  an  item  score  into  sevoral  subscoros  corresponds  effectively  to  an 
increase  in  tho  number  of  test  items.  Thus  tho  mean  square  betwoen  individuals 
increases  relative  to  tho  residual  mean  square  with  a corresponding  incroaso  in  the 
reliability  coefficient.  Wien  tho  3ccros  on  oach  item  are  distributed  among  the 
various  tasks  of  the  item,  tho  reliability  coefficient  rises  to  an  acceptable  value. 
Trie  problem  of  tho  low  reliability  of  a performance  te3t  appears  to  havo  boon  solved 
satisfactorily.  Another  advantage  of  subdividing  the-  score  of  an  item  into  several 
components  is  the  simplification  of  the  grading  procedure. 

Implications  for  Further  Research 

'Hie  low  correlation  coefficients  between  tho  laboratory  performance  tosts 
in  Physics  and  other  measures  of  aptitude  and  achievement  suggest  the  nood  for  in- 


Table  In- 


Hoyt Reliability 

of  Performance  Tests 

as  a Function  of 

the  Number 

cf  Itoas  - 

University 

of  Minnesota 

Course  & Quartor 

K No 

. of  Items 

No.  of  Scores 

Hoyt  r 

Physics  7,  Fall  1951 

67 

3 

3 

. P2 

1!  II 

It 

8 

8 

.59 

Spring  1952 

81 

P 

2 

.21 

• 1 tr 

ti 

7 

7 

.60 

" K.11  1952 

62 

10 

10 

.60 

ti 

10 

30 

.75 

Fr.ysics  la,  Ftll  1952 
Sample  1 

62 

6 

6 

.bh 

" « 

" 

6 

21 

.89 

rhysics  la.  Fall  1952 
Sample  1 

6h 

6 

6 

• t3 

H fl 

11 

6 

21 

•93 

vestigating  the  relationship  between  various  measures  of  laboratory  work. 

A number  of  poncil-paprr  tests  on  laboratory  work  in  elementary  physics 
have  been  developed.  Sevoral  of  the  tests  have  been  administered  during  1952-1953  to 
students  enrolled  In  three  different  physics  courses  at  the  University  of  Minnesota. 
The  results  arc  being  analyzed  and  the  relationships  between  the  pencil -paper  and 
performance  test  scores  are  under  investigation. 

Summary  and  Conclusions 

The  investigation  dealt  with  the  relationship  between  performance  tosts  in 
physics  and  other  measures  cf  scholastic  aptitude  and  achievement.  Students  with  and 


11. 


without  high  school  physics  were  compared  on  theoretical  and  performance  tests  in 
college  physics.  The  dependence  of  the  Hoyt  test  reliability  on  the  number  of  items 
and  the  number  of  sub-scores  was  explored. 

The  major  findings  of  the  study  were: 

1.  Students  with  high  school  physics  background  appeared  to  to 
superior  to  those  without  high  school  physics  on  conventional  measures  of  achieve- 
ment in  college  pnysics.  Howevor,  the  differences  on  the  performance  tests  in  the 
laboratory  were  probably  not  significant. 

2.  A reduction  of  the  theoretical  component  and  other  modifications 
of  a performance  test  result  in  a lower  correlation  coefficient  between  it  and  other 
measures  of  scholastic  aptitude  and  achievement  in  physics. 

a.  The  correlation  coefficient  between  the  first-quarter  por- 
formanco  tost  in  physics  and  ACE  va3  .16,  not  significantly  different  from  zero. 

b.  The  correlation  coefficient  between  the  performance  tost  and 
High  School  Rank  was  also  .16. 

c.  The  correlation  coefficient  between  the  performance  test  and 
the  final  grade  in  physics  was  .33,  significant  at  one  percent. 

3.  The  various  changes  in  the  administration  of  the  performance  tests 
resulted  in  small  but  desirable  changes  in  the  average  discrimination  and  difficulty 
indices. 

4.  The  Hoyt  reliability  depended  on  the  number  of  items  <n  a test. 

It  was  found  that  the  reliability  of  a 6-item  performance  test  could  be  raised  from 
.43  to  .93  by  using  21  eubscores. 

Paper-pencil  tests  on  laboratory  work  in  physics  have  been  constructed  and 
a number  of  them  have  been  administered.  Their  relation  to  performance  te3ts  is 
being  investigated. 


12. 
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